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States  and 
transformations 


States  and 
transformations 
describe  physical 
variation  within  an 
object  class. 
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unripe 


rotten 


Dataset 

Image  collection  for 
each  noun,  depicting 
physical  state 
variation  that  noun 
can  undergo. 


249  Nouns 
115  Adjectives 
70k  images 


Adjectives  and  antonyms 


We  define  a  state  as  an 
adjective,  and  a 
transformation  as  a 
transition  from  a  one 
adjective  to  its  antonym. 


Draped 
Loose  <=>  Tight 
Heavy  <=>  Lightweight 
Crumpled 
Crinkled 


Unripe  <=>  Ripe 
Raw  <=>  Cooked 
Burnt 
Mashed 
Pureed 


Sunny  <=>  Foggy  Engraved 

Murky  <=>  Clear  Broken 

Sunny  <=>  Cloudy  Narrow  <=>  Wide 
Windblown  Ancient  <^>  Modern 

Muddy  <=>  Dry  Curved  <=>  Straight 


Adjectives  modify  different  images  in 
similar  ways 

Melted  chocolate  Melted  sugar  Melted  butter 


C.f.  Beyond  nouns:  Exploiting  prepositions  and  comparative  adjectives  for  learning  visual  classifiers.  Gupta  &  Davis,  ECCV,  2008 
Describing  objects  by  their  attributes,  CVPR  2009 
Relative  Attributes.  Parikh  &  Grauman,  ICCV  201 1 

Constrained  Semi-Supervised  Learning  using  Attributes  and  Comparative  Attributes.  Shrivastava  et  al. ,  ECCV  2012 


Tasks 


Input:  Tomato 


Never  before  seen  image  class 


1 .  Identify  states 


2.  Rank  images  by 
transformations 
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Given  a  novel  image 
class,  we  parse  the 
intra-class  variation 
into  a  set  of  states 
and  transformations 
depicted  in  that 
photos  of  that  class. 


Collection  parsing 
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Using  collection 
helps 


Results 


State  and  transformation  discovery 


Chocolate 

Molten 

Caramelized 

Burnt 

Whipped 

Crushed 

Jacket 


Draped 
Loose  <=>  Tight 
Heavy  <^>  Lightweight 
Crumpled 
Crinkled 


Beach 


Sunny  <=>  Foggy 
Murky  <^>  Clear 
Sunny  <=>  Cloudy 
Windblown 
Muddy  <=>  Dry 


Ranking  images  according  to  a  transformation 


Antonyms  pin  down  a  meaningful  dimension  of  variation 


Street 

Desk 


Cluttered  < - ►  Empty 


How  well  can  the  system  generalize? 
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Old  Computer  < - >  New  Computer 


Cluttered  Room  <  >  Empty  Room 
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